1. Field of the Invention
The present invention relates to an image capturing apparatus and control method thereof.
2. Description of the Related Art
As an image sensor used in an image capturing apparatus, a CCD (Charge Coupled Device) type image sensor (to be referred to as “CCD sensor” hereinafter) is generally adopted. However, in recent years, as image sensors are required to have a larger number of pixels, a CMOS (Complementary Metal Oxide Semiconductor) type image sensor (to be referred to as “CMOS sensor” hereinafter) has received a lot of attention.
In an image sensor such as a CMOS sensor, photoelectric conversion elements of pixels perform photoelectric conversion for accumulating charges according to incident light amounts, and outputting electrical signal corresponding to the accumulated charges. Also, an image sensor such as a CMOS sensor has an electronic shutter function. The electronic shutter function starts exposure by resetting photoelectric conversion elements of pixels, and ends exposure by reading out charges accumulated on the photoelectric conversion elements. In this manner, since the start and end of exposure are controlled by only the function of the image sensor, an exposure time from a low-speed shutter to a high-speed shutter can be accurately controlled.
Furthermore, as one of features of a CMOS sensor, a rolling shutter operation (also called a focal plane shutter operation) is known. With the rolling shutter operation, charges on pixels are reset by sequentially scanning a plurality of two-dimensionally arranged pixels for each line unlike in a CCD sensor. Then, after an elapse of a predetermined exposure time, the pixels are sequentially scanned for each pixel to read out accumulated charges and to output signals.
In this manner, the rolling shutter operation has a time difference required to read out charges and to output signals for each line. Thus, exposure times are deviated for respective lines in a single image capturing operation.
An image capturing apparatus using an image sensor such as a CMOS sensor suffers a problem that a dynamic range is normally insufficient upon capturing an image of an object including both a bright part and dark part. For example, when an exposure time is controlled to be short in correspondence with a bright part, since a sufficient exposure time for a dark part cannot be assured, a shadow-detail loss and an image quality drop caused by deterioration of an S/N are generated. Conversely, when an exposure time is controlled to be long in correspondence with a dark part, accumulated charge amounts of some photoelectric conversion elements reach a saturation level, thus generating a highlight-detail loss in which an object region having a given brightness or higher is set at a saturated luminance level.
As a method of accurately reproducing tones of bright and dark parts, dynamic range expansion processing is known. The dynamic range expansion processing attains a high S/N by controlling an exposure time to be long for pixels with smaller incident light amounts on an image sensor and avoids saturation for pixels with larger incident light amounts.
As one method of the dynamic range expansion processing, a multi-exposure method for successively capturing a plurality of images with different exposure times by a single image sensor and composing these images is known. With the multi-exposure method, a long-term exposure image and short-term exposure image are successively and individually shot. Then, composition processing is executed using the long-term exposure image for a dark image region and the short-term exposure image for a bright image region which may cause a highlight-detail loss in the long-term exposure image. In this manner, one image, a dynamic range of which is expanded, is generated.
However, in the multi-exposure method, long-term exposure and short-term exposure have to be alternately performed, and signals obtained from the respective exposures have to be alternately read out. When the multi-exposure method is applied to a moving image shooting operation, one frame period upon long-term exposure and that upon short-term exposure have to be equal to each other so as to set a uniform frame rate. Then, a maximum exposure time of the long-term exposure has to be set to be one frame period, and an S/N enhancement effect of the long-term exposure cannot be expected since a maximum exposure time in a normal operation of the image sensor can also be set to be one frame period. Furthermore, when an exposure time ratio is 2:1, since a maximum exposure time of the short-term exposure is set to be half of one frame period, a wasteful time which does not contribute to exposure is generated.
Also, in the multi-exposure method, a time difference for one frame period is generated between the long-term exposure and short-term exposure. For this reason, when a moving object is included, a long-term exposure image and short-term exposure image cannot be the same. As a result, in an image which has undergone the dynamic range expansion processing, the moving object may be blurred and false colors may be generated.
In order to solve these problems of the multi-exposure method, other dynamic range expansion processing methods are disclosed by Japanese Patent Laid-Open Nos. 2012-105225 and 2011-244309. In Japanese Patent Laid-Open Nos. 2012-105225 and 2011-244309, different exposure times are set for pixels which are divided into two groups at 2-line intervals. Then, for example, high-sensitivity pixel information is acquired from a long-term exposure pixel, and low-sensitivity pixel information is acquired from a short-term exposure pixel, thus generating an image, a dynamic range of which is expanded, based on these pieces of pixel information of different sensitivities. According to this method, an image, a dynamic range of which is expanded, can be generated based on one shot image.
Japanese Patent Laid-Open No. 2011-244309 has also proposed a method of eliminating a blur of a moving object as a problem of the multi-exposure method by matching the centers of exposure times of the long-term exposure and short-term exposure (FIG. 13 of Japanese Patent Laid-Open No. 2011-244309).
However, in a method described in FIG. 6 of Japanese Patent Laid-Open No. 2012-105225 or FIG. 9 of Japanese Patent Laid-Open No. 2011-244309, all lines of signals of pixels having different exposure times at 2-line intervals are read out at the same time. For this reason, correction processing and signal processing for long-term exposure pixels and those for short-term exposure pixels have to be switched and performed at 2-line intervals. Such processing imposes a heavy load on a signal processing circuit.
The method described in FIG. 13 of Japanese Patent Laid-Open No. 2011-244309 requires that the center of the short-term exposure always matches that of the long-term exposure. Then, every time reset and read timings of the long-term exposure are changed due to a change in exposure time, reset and read timings have to be changed to match the center of the short-term exposure. Therefore, such method requires complicated timing control.